Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create mongo key file securely #1232

Merged
merged 31 commits into from
Jun 15, 2021
Merged

Create mongo key file securely #1232

merged 31 commits into from
Jun 15, 2021

Conversation

shreyamalviya
Copy link
Contributor

What does this PR do?

Fixes #1195

PR Checklist

  • Have you added an explanation of what your changes do and why you'd like to include them?
  • Is the TravisCI build passing?
  • Was the CHANGELOG.md updated to reflect the changes?
  • Was the documentation framework updated to reflect the changes?

Testing Checklist

  • Added relevant unit tests?
  • Have you successfully tested your changes locally? Elaborate:

    Tested by {Running the Monkey locally with relevant config/running Island/...}

  • If applicable, add screenshots or log transcripts of the feature working

Explain Changes

Are the commit messages enough? If not, elaborate.

monkey/monkey_island/cc/environment/utils.py Outdated Show resolved Hide resolved
flags = (
os.O_RDWR | os.O_CREAT | os.O_EXCL
) # read/write, create new, throw error if file exists
mode = 0o700 # read/write/execute permissions to owner
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should use stat.S_IRWUX here instead of a magic number.

@@ -29,11 +30,9 @@ def _create_secure_directory_linux(path: str):
# Don't split directory creation and permission setting
# because it will temporarily create an accessible directory which anyone can use.
os.mkdir(path, mode=0o700)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should use stat.S_IRWUX here instead of a magic number.

Copy link
Contributor

@VakarisZ VakarisZ Jun 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That "magic number" is the standard unix permission expression, when you see 700 it's clear that it's: owner (read + write + execute), group (none), public (none). What S_IRWUX means heaven only knows.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mistyped it. It's S_IRWXU which is for Read Write eXecute User. It's very commonly seen in code written for Linux.

I'm fine with either, but let's be consistent. We're using the constatns from stats in a test, but the magic number elsewhere.

def test_create_secure_file__already_created(test_path):
os.close(os.open(test_path, os.O_CREAT, 0o700))
assert os.path.isfile(test_path)
create_secure_file(test_path)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test isn't asserting anything after create_secure_file(), which makes it unclear what it's testing.

I think it's relying on the fact that if an exception is thrown, the test will fail. In essence, it's testing that no exceptions are thrown. Maybe a comment is in order. # test will fail if any exceptions are thrown or something similar that explains the intent of the test.

@@ -21,6 +23,7 @@ def __init__(self, password_file_dir):
if os.path.exists(password_file):
self._load_existing_key(password_file)
else:
create_secure_file(path=password_file)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe if create_secure_file() returns the file handler without closing, you could do something like:

with open(create_secure_file(password_file), "wb"):

This would avoid the os.close(os.open(...)) ugliness in the create_secure_file() function. You should probably then rename it to open_new_secure_file() or something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What purpose would that serve? We would just be doing something like

with open(create_secure_file(password_file), "wb"):
    pass

?

Plus, I don't mind the os.close(os.open(...)) ugliness if it means that we're maintaining consistency in utils.py (where we're simply creating secure files/directories, not opening them).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are better ways to improve readability. Creating and opening and then closing the file is not a very intuitive workflow just for creating a file.

Copy link
Collaborator

@mssalvatore mssalvatore Jun 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You wouldn't use open() and then pass. The code that used it would look like this:

diff --git a/monkey/monkey_island/cc/server_utils/encryptor.py b/monkey/monkey_island/cc/server_utils/encryptor.py
index abeb34dc..9ec3dc37 100644
--- a/monkey/monkey_island/cc/server_utils/encryptor.py
+++ b/monkey/monkey_island/cc/server_utils/encryptor.py
@@ -6,7 +6,7 @@ import os
 from Crypto import Random  # noqa: DUO133  # nosec: B413
 from Crypto.Cipher import AES  # noqa: DUO133  # nosec: B413
 
-from monkey_island.cc.server_utils.file_utils import create_secure_file
+from monkey_island.cc.server_utils.file_utils import open_new_secure_file
 
 __author__ = "itay.mizeretz"
 
@@ -23,12 +23,11 @@ class Encryptor:
         if os.path.exists(password_file):
             self._load_existing_key(password_file)
         else:
-            create_secure_file(path=password_file)
             self._init_key(password_file)
 
     def _init_key(self, password_file):
         self._cipher_key = Random.new().read(self._BLOCK_SIZE)
-        with open(password_file, "wb") as f:
+        with open(open_new_secure_file(password_file, "wb")) as f:
             f.write(self._cipher_key)
 
     def _load_existing_key(self, password_file):

Rather than close(open()) and then open() the file again later, you would just open the file and write to it. Open a file and write to it is the standard workflow for handling files. Create a directory and use it is the standard workflow for directories. There's not really an inconsistency.

You could consider names like get_file_descriptor_for_secure_file() instead of open_secure_file() or something else. The point is, we should just open a file and write to it.

@ghost
Copy link

ghost commented Jun 15, 2021

DeepCode's analysis on #44bdfa found:

  • ⚠️ 1 warning, ℹ️ 1 minor issue. 👇

Top issues

Description Example fixes
Use os.makedirs instead of os.mkdir because the given path may require creating the parent directories. Occurrences: 🔧 Example fixes
Use os.makedirs instead of os.mkdir because the given path may require creating the parent directories. Occurrences: 🔧 Example fixes

👉 View analysis in DeepCode’s Dashboard | Configure the bot

👉 The DeepCode service and API will be deprecated in August, 2021. Here is the information how to migrate. Thank you for using DeepCode 🙏 ❤️ !

If you are using our plugins, you might be interested in their successors: Snyk's JetBrains plugin and Snyk's VS Code plugin.

os.O_RDWR | os.O_CREAT | os.O_EXCL
) # read/write, create new, throw error if file exists
mode = 0o700 # read/write/execute permissions to owner
os.close(os.open(path, flags, mode))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we even need the complexity of the flags? Why not simply Path(path).touch(0o700)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice; changed it.

Comment on lines 76 to 78
file_sharing = (
win32file.FILE_SHARE_READ
) # subsequent open operations on the object will succeed only if read access is requested
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange, why do we need () here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's how black formats it. Because it's in the next line.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That means the comment needs to be above the statement and the statement needs to be one like, like so: file_sharing = win32file.FILE_SHARE_READ

Comment on lines 94 to 96
win32job.CreateJobObject(
None, ""
), # https://stackoverflow.com/questions/46800142/in-python-with-pywin32-win32job-the-createjobobject-function-how-do-i-pass-nu # noqa: E501
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No clue about what win32job.CreateJobObject does, why do we call it here, what argument it specifies etc. The whole win32file.CreateFile function should be called with keyword arguments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, just looks like a dirty workaround, there should be an easier way to pass Null. Either way, this should be hidden in a clearly named function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole win32file.CreateFile function should be called with keyword arguments.

Can't. It throws an error that it can't be called with keyword arguments.

Generally, just looks like a dirty workaround, there should be an easier way to pass Null.

I agree. Couldn't find an easier way, though. I'll try again, perhaps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's OK if that's the only way we can provide NULL argument, but you should create a method called get_null_value_for_win32 or something and call it instead. Then it would be clear that we actually provide NULL as the last argument and not some kind of created job.

Lastly, don't put comments that obviously violate line length inline, put them above the statement.

windows_permissions.get_security_descriptor_for_owner_only_perms()
)
file_creation = win32file.CREATE_NEW # fails if file exists
file_attributes = win32file.FILE_FLAG_BACKUP_SEMANTICS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this flag?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using win32file.FILE_ATTRIBUTE_NORMAL throws an access denied error. win32file.FILE_FLAG_BACKUP_SEMANTICS doesn't probably because of:

The operating system ensures that the calling process overrides file security checks, provided it has the necessary permission to do so.

(Source: http://timgolden.me.uk/pywin32-docs/win32file_FILE_FLAG_BACKUP_SEMANTICS.html)

Copy link
Contributor

@VakarisZ VakarisZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address comments

@mssalvatore
Copy link
Collaborator

The permissions the mongo file should be 600. We don't need the execute bit set on this file.

…_file()

get_file_descriptor_for_new_secure_file() should return a file
descriptor. If the file already exists, this function would return
nothing, potentially causing issues with whatever relies on this
function's output.
create_secure_file() was previously renamed to
get_file_descriptor_for_new_secure_file().
@mssalvatore mssalvatore merged commit 78e9b8c into develop Jun 15, 2021
@mssalvatore mssalvatore deleted the secure-mongo-key-file branch June 15, 2021 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create mongo key file with secure permissions
3 participants